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Abstract 

We provide a dynamic programming principle for stochastic optimal 
control problems with expectation constraints. A weak formulation, 
using test functions and a probabilistic relaxation of the constraint, 
avoids restrictions related to a measurable selection but still implies 
the Hamilton-Jacobi-Bellman equation in the viscosity sense. We treat 
open state constraints as a special case of expectation constraints and 
prove a comparison theorem to obtain the equation for closed state 
constraints. 
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1 Introduction 

We study the problem of stochastic optimal control under state constraints. 
In the most classical case, this is the problem of maximizing an expected 
reward, subject to the constraint that the controlled state process has to 
remain in a given subset O of the state space. There is a rich literature 
on the associated partial differential equations (PDEs), going back to |13l 
[HI [U [TO] [T7] in the first order case and \12\ [IT] [TO] in the second order 
case. The connection between the control problem and the equation is given 
by the dynamic programming principle. However, in the stochastic case, it 
is frequent practice to make this connection only formally, and in fact, we 
are not aware of a generally applicable, rigorous technique in the literature. 
Of course, there are specific situations where it is indeed possible to avoid 
proving the state-constrained dynamic programming principle; in particular, 
penalization arguments can be useful to reduce to the unconstrained case 
(e.g., We refer to [6] [18] for further background. 
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Generally speaking, it is difficult to prove the dynamic programming 
principle when the regularity of the value function is not known a priori, due 
to certain measurable selection problems. It was observed in [3] that, in the 
unconstrained case, these difficulties can be avoided by a weak formulation 
of the dynamic programming principle where the value function is replaced 
by a test function. This formulation, which is tailored to the derivation of 
the PDE in the sense of viscosity solutions, avoids the measurable selection 
and uses only a simple covering argument. It turns out that the latter does 
not extend directly to the case with state constraints. Essentially, the reason 
is that if v is some admissible control for the initial condition x € O — i.e., 
the controlled state process X" x started at x remains in O — then v may fail 
to have this property for a nearby initial condition x' £ O. 

However, if O is open and mild continuity assumptions are satisfied, then 
XY , will violate the state constraint with at most small probability when 
x' is close to x. This observation leads us to consider optimization problems 
with constraints in probability, and more generally expectation constraints 
of the form E\g(X¥ x {T))\ < m for given m 6 R. We shall see that, following 
the idea of [2] , such problems are amenable to dynamic programming if the 
constraint level m is formulated dynamically via an auxiliary family of mar- 
tingales. A key insight of the present paper is that relaxing the level to by 
a small constant allows to prove a weak dynamic programming principle for 
general expectation constraint problems (Theorem 12. 4p . while the PDE can 
be derived despite the relaxation. We shall then obtain the dynamic pro- 
gramming principle for the classical state constraint problem (Theorem I3.ip 
by passing to a limit m J. 0, with a suitable choice of g and certain regularity 
assumptions. Of course, expectation constraints are of independent interest 
and use; e.g., in Mathematical Finance, where one considers the problem of 
maximizing expected terminal wealth E[Xfi X (T)] under the loss constraint 
E[([Xq x (T) — x]~) p ] < to, for some given m,p > 0. 

We exemplify the use of these results in the setting of controlled diffusions 
and show how the PDEs for expectation constraints and state constraints 
can be derived (Theorems 14.21 and I4.6[) . For the latter case, we introduce an 
appropriate continuity condition at the boundary, under which we prove a 
comparison theorem. While the above concerned an open set O and does not 
apply directly to the closed domain O, we show via the comparison result 
that the value function for O coincides with the one for O, under certain 
conditions. 

The remainder of the paper is organized as follows. In Section [2] we 
introduce an abstract setup for dynamic programming under expectation 
constraints and prove the corresponding relaxed weak dynamic programming 
principle. In Section[3]we deduce the dynamic programming principle for the 
state constraint O. We specialize to the case of controlled diffusions in Sec- 
tion [U where we study the Hamilton- Jacobi-Bellman PDEs for expectation 
and state constraints. Appendix lAl provides the comparison theorem. 
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Throughout this paper, (in) equalities between random variables are in 
the almost sure sense and relations between processes are up to evanescence, 
unless otherwise stated. 

2 Dynamic Programming Principle for Expectation 
Constraints 

We fix a time horizon T G (0, oo) and a probability space (Q, F, P) equipped 
with a filtration F = (J~t)t€[o,T] ■ For each t G [0, T], we are given a set lAt 
whose elements are seen as controls at time t. Given a separable metric space 
S, the state space, we denote by S := [0, T] x S the time-augmented state 
space. For each (t, x) G S and v G lit, we are given a cadlag adapted process 
X" x = {X" x (s), s G [t,T]} with values in S, the controlled state process. 
Finally, we are given two measurable functions /, g : 5 — > R. We assume 
that 

E[\f(X» x (T))\] < oo and E[\g(X^ x (T))\] < oo for all u G U t , (2.1) 
so that the reward and constraint functions 

F(t,x;u) := E\f(X^(T))], G(t,x;u) := E[g{Xl x (T))} 
are well defined. We also introduce the value function 

V(t,x,m) := sup F(t,x;v), (t,x,m)&S, (2-2) 

v&U{t,x,m) 

where S:=Sx1e[0,T] xSxM and 

U(t, x, m) := {u eU t : G(t,x;u) < m} (2.3) 

is the set of controls admissible at constraint level m. Here sup0 := — oo. 

The following observation is the heart of our approach: Given a control 
v admissible at level m at the point x, relaxing the level m will make v ad- 
missible in an entire neighborhood of x. This will be crucial for the covering 
arguments used below. We use the acronym u.s.c. (l.s.c.) to indicate upper 
(lower) semicontinuity. 

Lemma 2.1. Let (t,x,m) G S, v G U(t,x,m) and assume that the function 
x' i— > G(t, x'; v) is u.s.c. at x. For each 5 > there exists a neighborhood B 
of x G S such that v G U(t, x', m! + 5) for all x' G B and all m' > m. 

Proof. We have G(t, x\v) < m by the definition (|2.3p of U(t, x, m). In view 
of the upper semicontinuity, there exists a neighborhood B of x such that 
G(t,x'; v) < m + 5 < m' + 5 for x' G B; that is, v G U(t,x',m' + 5). □ 
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A control problem with an expectation constraint of the form (|2.3p is 
not amenable to dynamic programming if we just consider a fixed level m. 
Extending an idea from [21 [1], we shall see that this changes if the con- 
straint is formulated dynamically by using auxiliary martingales. To this 
end, we consider for each t G [0, T] a family M-t,0 of cadlag martingales 
M = {M(s), s G [t,T]} with initial value M(t) = 0. We also introduce 

M t , m ■= {™ + M : M G M t ,o}, m G R. 

We assume that, for all (t, x) G S and v EUt, 

there exists M?[x] G M t , m such that M?[x](T) = g(X£ x (T)), (2.4) 

where, necessarily, m = E[g(X^ x (T))]. In particular, given v G Ut, the set 

Mf^M ■= {M G M t>m : M(T) > g(X£ x (T))} 

is nonempty for any m > E[g(X^ x (T))]. More precisely, we have the follow- 
ing characterization. 

Lemma 2.2. Let (t,x) G S and m G R. T/ien 

W(t,s,m) = {i/ GW t : Mf^i/) ^ 0}. 

Proof. Let u Elif If there exists some M G -M^" m j. (f) , then taking expec- 
tations yields E[g(X£ x (T))] < E[M(T)] = m and hence v G U(t, x, m). 

Conversely, let v G U(t,x,m); i.e., we have m! := E[g(X^ x (T))] < m. 
With Mf[x] as in ([13]), M := M^[x]+m-m' is an element of M £ m>a . (i/) . □ 

It will be useful in the following to consider for each t G [0, T] an auxiliary 
subfiltration F* = (J~t)se[o,T\ °f ^ such that X" x and M are F*-adapted for 
all x G S", v G ZYt and M G A^^o- Moreover, we denote by T* the set of 
F*-stopping times with values in [t,T]. 

The following assumption corresponds to one direction of the dynamic 
programming principle; cf. Theorem I2.4f i) below. 

Assumption A. For all (t,x,m) G S, v G U(t,x,m), M G M.f mx (u), 
t G T* and P-a.e. uj G 0,, there exists v w G U(t(uj), X^ x {t){u), M(t)(uj)) 
such that 

£[/(X t yT))|.F r ](u,) < F(rH,I ( yT)H;^). (2.5) 

Next, we state two variants of the assumptions for the converse direction 
of the dynamic programming principle; we shall comment on the differences 
in Remark 12.51 below. In the first variant, the intermediate time is deter- 
ministic; this will be enough to cover stopping times with countably many 
values in Theorem I2.4f ii) below. We recall the notation M^[sc] from (|2.4p . 
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Assumption B. Let (t, x) G S, v G lit-, s G [i, T] , v &U S and T G J^. 
(Bl) There exists a control v (zUt such that 

X t y.)=^x(0 on[t,T]x(0\r); (2.6) 

XZ X (.) = Xl x „ Js) (.) on[ S ,T]xr ; (2.7) 

£?[/(X^(T))|Jv,] > F(s,Xl x {s)-v) on T. (2.8) 

The control v is denoted by v 8)( S) r) ^ an d called a concatenation of i/ 
and z/ on (s, T). 

(B2) Let M G .Mto- There exists a process M = {M(r), r G [s, T]} such 
that 

M(-)M = (Mf[^( S )(a;)](-))(a;) on [ S ,T] P-a.s. 

and 

Ml [t , s) + l [SjT ] (Ml nvr + [M - M(s) + Af (a)] l r ) G M,o- 
(B3) Let m G E and M G M-tm,xi u )- For P-a.e. well, there exist a control 

In the second variant, the intermediate time is a stopping time and we 
have an additional assumption about the structure of the sets IA S . This 
variant corresponds to Theorem I2.4f ii') below. 

Assumption B\ Let (t,x) G S, v G U u r G T*, T G J* and v G W|| r || i( x.- 
(BO') U s D W s / for all < s < s' < T. 

(Bl') There exists a control u G tit, denoted by v ®( r ,r) such that 

*£x(0 on[t,T]x(0\r); 

= ^.wCO on I T ' ^1 x r; (2.9) 

S[/(X t %(T))|j=- T ] > F(T,X t ^(r);P) on I\ (2.10) 

(B2') Let M G M t)0 . There exists a process M = {M(r), r G [r,T]} such 
that 

M(.)H = (Af T % ) [X^,(r)H](.))M on [r,T] P-a.s. 

and 

Ml [tjT) + l [TjT] (m 1 q \p + [M - M(r) + M (r)] l r ) G M,o- 

(B3') Let m G E and M G -M/~ m x (i/). For P-a.e. well, there exist a control 
v w EU{t{u),XI x {t){u)\m{t){uj)). 
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Remark 2.3. (i) Assumption [B^ implies Assumption |Bj Moreover, As- 
sumption [A] implies (E(3j). 

(ii) Let D := {(t,x,m) € S : U(t,x,m) 7^ 0} denote the natural domain 
of our optimization problem. Then (EO) can be stated as follows: for 
any v G U(t, x, m) and M G -M-tm x( v )i 

{t,XI x (t),M{t)) G D P-a.s. for all tG T*. (2.11) 

Under (E(3]) , the same holds if r takes countably many values. We 
remark that the invariance property (|2.1ip corresponds to one direction 
in the geometric dynamic programming of |15| . 

(hi) We note that (|2.9p (and similarly (12. 7p ) states in particular that 

(r, W ) ^ (X^ x{T) (r))(u) := (i; H ,,y T)H (r))(.) 

is a well-defined adapted process (up to evanescence) on [i, T] x T. Of 
course, this is an implicit measurability condition on (i, x) 1— > X" x . 

(iv) For an illustration of (E[U), let us assume that the controls are pre- 
dictable processes. In this case, one can often take 

v ® (T)r j v := z/l [0)T] + l (TjT] (z/l r + ul Q \ r ) (2.12) 

and the condition that v <8>( Tj r) ^ € is called stability under concate- 
nation. The idea is that we use the control v up to time r; after time 
t, we continue using v in the event Vt \ T while we switch to the control 
v in the event T. Of course, one can omit the set r 6 J> by observing 
that 

v ®( r ,r) v = ^1[o,t'] + ^!(t',T] 
for the F*-stopping time r' := rip + Tlmp. 

We can now state our weak dynamic programming principle for the 
stochastic control problem fj2.2j) with expectation constraint. The formu- 
lation is weak in two ways; namely, the value function is replaced by a 
test function and the constraint level m is relaxed by an arbitrarily small 
constant 5 > (cf. (|2,15p below). The flexibility of choosing the set T> ap- 
pearing below, will be used in Section [3] We recall the set D introduced in 
Remark I2.3f ii). 

Theorem 2.4. Let (t,x,m) G S, v G U(t,x,m) and M G M.f mx {u). Let 
t GT 1 and letV C S be a set such that (t,X£ x (t),M(t)) G V holds P-a.s. 

(i) Let Assumption^ hold true and let ip : S — > [— 00,00] be a measurable 
function such that V < tp onT>. Then E[tp(r, X^ x (t), M(t))~] < 00 
and 

F(t,x;u) < E[<p(r,X^(T),M(r))]. (2.13) 
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(ii) Let 5 > 0, let Assumption^ hold true and assume that r takes count- 
ably many values (ij)j>i. Let <p : S — y [—00,00) be a measurable func- 
tion such that V > if onT>. Assume that for any fixed v 6 H^, 

(x',m') 1 — y ip(t{,x' ,m') is u.s.c. 

x' F(ti,x';u) isl.s.c. ) onV 1 (2.14) 
x' 1 — y G(ti,x' \v) is u.s.c. 

for all i > 1, where V i := {{x',m') : (t h x' ,m') £ T>} <Z S x R. Then 

V(t,x,m + S)>E[ip(T,Xl x (r),M(T))]. (2.15) 

(ii') Let 5 > and let Assumption\B^ hold true. Let (f : S — >■ [—00,00) be a 
measurable function such that V > (p on T>. Assume that T> n D C S 
is a -compact and that for any fixed v € Ut , to € [£, T] ; 

(t',x',m') i-> ip(t' , x' ,m') is u.s.c. 

{t',x') ^y F(t',x';P) isl.s.c. )■ on V D {t' < t } . (2.16) 
(t',x f ) 1 ^ G(t',x';z/) is u.s.c. 

TTien (|2.15p /io/c/s true. 

The following convention is used on the right hand side of (|2.15p : if Y is 
any random variable, 

we set E[Y] := -00 if E[Y + ] = E[Y~] = 00. (2.17) 

We note that Assumption (E(UJ) ensures that the expressions F{t\x'\u) and 
G(t',x'; v) in (f2~T6|) are well defined for t' < t . 

Remark 2.5. The difference between parts (ii) and (ii') of the theorem 
stems from the fact that in the proof of (ii') we consider [0, T] x S x K 
as the state space while for (ii) it suffices to consider S x R and hence no 
assumptions on the time variable are necessary. Regarding applications, (ii) 
is obviously the better choice if stopping times with countably many values 
(and in particular deterministic times) are sufficient. 

There is a number of cases where the extension to a general stopping 
time r can be accomplished a posteriori by approximation, in particular if 
one has a priori estimates for the value function so that one can restrict to 
test functions with specific growth properties. Assume for illustration that 
/ is bounded from above, then so is V and one will be interested only in test 
functions ip which are bounded from above; moreover, it will typically not 
hurt to assume that cp is u.s.c. (or even continuous) in all variables. Now 
let (r n ) be a sequence of stopping time taking finitely many (e.g., dyadic) 
values such that r n \. r P-a.s. Applying (|2.15p to each r n and using Fatou's 
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lemma as well as the right-continuity of the paths of X" x and M, we then 
find that f|2. 15j) also holds for the general stopping time r. 

On the other hand, it is not always possible to pass to the limit as above 
and then (ii') is necessary to treat general stopping times. The compactness 
assumption should be reasonable provided that S is cr-compact; e.g., S = M. d . 
Then, the typical way to apply (ii') is to take (t,x,m) £ intD and let T> be 
a open or closed neighborhood of (t,x,m) such that PCD. 

Proof of Theorem \2.4\ (i) Fix (t,x,m) € S. With v w as in (|2.5p . the defini- 
tion (|2.2p of V and V < (f on T> imply that 

E[f(X^(T))\JF T ]( U ) < F(r( U ) 3 X^(T)(u);u u ) 

<V{t(u),X?Jt)(u,),M(t)(u,)) 

< <p(T(u),X^(r)(u),M(T)(u)) P-a.s. 

After noting that the left hand side is integrable by (|2.ip . the result follows 
by taking expectations. 

(ii) Fix (t, x, m) € S and let e > 0. 

1. Countable Cover. Like V, the set T> fl D has the property that 

(T,^(r),M(T))eUnD P-a.s.; (2.18) 

this follows from (|2.1ip since r takes countably many values. Replacing T> 
by T> fl D if necessary, we may therefore assume that T> C D. Using also 
that V > if on T>, the definition ()2.2p of V shows that there exists for each 
(s, y, n) € P some v s ' y,n G W(s, y, n) satisfying 

F( S ,y;^")>^( S ,y,n)- e . (2.19) 

Fix one of the points t{ in time. For each (y,n) € the semicontinuity 
assumptions ()2.14p and Lemma 12.11 imply that there exists a neighborhood 
(y, n) € B l (y, n) C S 1 := 5 x R (of size depending on y, n, i, e, 5) such that 

u U,y,n e u(t h y',ri + 8) 1 
tp(ti,y',ri) < ip(ti,y,n)+e \ for all (y',n') € B i (y,n)nV i . 
F(t h y';^ n ) > F(ti,y;^y' n )-£ J 

(2.20) 

Here the first inequality may read — oo < — oo. We note that T> % C 5 is metric 
separable for the subspace topology relative to the product topology on S. 
Therefore, since the family {B l (y,n) (I'D 1 : (y,n) G /S} forms an open cover 
of D l , there exists a sequence (yj,nj)j>i in 5 such that {B l (yj,rij) H T> l }j>i 
is a countable subcover of D\ We set v % - := v ti,y i> n i and i?*- := B l (yj,rij), so 
that 

P'cy B). (2.21) 
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We can now define, for i still being fixed, a measurable partition (A* ) J >i of 
U^iSj by 

4--=5i, A} +1 :=B} +1 \(Biu...UJ3}). i> L 
Since A 1 - C Bj, the inequalities (f2~T9]) and (f2720|) yield that 

F{U, y'- uj) > <p(U,y', ri) - 3e for all (y', n') G n (2.22) 

5. Concatenation. Fix an integer fc > 1; we now focus on (ti)i<j<fc. We 
may assume that t\ < t2 < ■ ■ ■ < tfc, by eliminating and relabeling some of 
the tj. We define the J-^-measurable sets 

V) := {t = U and (X» X (U), M{U)) G A)} G J% and T(k) := (J r§. 

l<ij'<fe 

Since the ij are distinct and ^4*- n A*-/ = for j ^ f, we have T*- n T*', = 
for ^ (i',f). We can then consider the successive concatenation 

u(k) := v ® (tlj ri) W ®(ti,ri) ^2 ■ ■ ■ ®(tij-i) ^fc ®(ta,r?) ^ " ' ®(t*,r$) 

which is to be read from the left with v a ® z/ 6 ® z> c := (z/ 1 <8> zv b ) ® z/ c . It 
follows from an iterated application of (EQ]) that v{k) is well defined and in 
particular v{k) G lit- (To understand the meaning of v(k), it may be useful 
to note that in the example considered in (|2.12p . we would have 

u(k) = z/l [0)T] + 1( T)T] ( ul n \ T{k) + ^ z/]lpi J ; 

i.e., at time r, we switch to z/j on T*-.) We note that fj2.6j) implies that 
= Jf^ onO\ r(jfe) G and hence 

E[/(X^ fc) (T))|j- r ] =E[f(X^ x (T))\T T ) onn\r(fc). (2.23) 

Moreover, the fact that r = ij on T*-, repeated application of (|2,6p . and (|2.8p 
show that 

E[f(X& k) (T))\jF T ] > F{t,XI x {t);v)) on rj, for 1 < i,j < k, 
and we deduce via (|2.22p that 

E[f(X^\T))\F T ]l T{k) > £ F(r,X^(r);^)l r , 

l<i,i<fc 

> £ {ip(t,X» x (t),M(t)) -3e)l r] 

l<i,j<k 

> ip(T,Xl x (T),M(r))l m -3e. (2.24) 



9 



3. Admissibility. Next, we show that v(k) £ U(t,x,m + 5). By (E(2]) 
there exists, for each pair 1 < i,j < k, a process M % - = {M l -{u), u G [ij,T]} 
such that 

Mj(-)(u) = (M$ [*£,(*)(«)](■))(*) on r* (2.25) 

and such that 
M (t) := (M + S)l [ttT) 

+ l [T J(M + 5)l n \ r{k) + [Mi-MjM+MM + Sllri 

l<i,j<fc 

is an element of M t ,m+S- We note that M^>(T) > Mj(T) on Y) since 
Mj(tj) < M(ti) + 5 on rj as a result of the first condition in ([2720]) . Hence, 
using (f2TT|) and ([2725]) . we have 

9 (^i fc) (T)) = ff (X^ Xt%(tj) (T)) < Mj(T) < M«(T) on I*. (2.26) 

This holds for all 1 < i,j < k. On the other hand, using (|2.6p and that 
M € Mf mx (u), we have that 

<?(xg fc) (T)) = g(Xl x (T)) < M{T) < M{T) + 5 = M^(T) on Q \ F(k). 

(2.27) 

Combining ([Z5SD and (l2~27D . we have g(Xt£\T)) < AfW(T) on and so 
Lemma 12.21 yields that z^(&) eW(t,i,m|(5), 

^. e-Optimality. We may assume that either the positive or the negative 
part of <p(r, X^ x (t), M(t)) is integrable, as otherwise our claim (|2. 15[) is 
trivial by (12TTD . Using the definition $272$ of V as well as and (f2T2lh 

we have that 

m + 5) > E [f(X%j® (T))] 

= £[£[/(x^))|^V]] 

> E[<p(r,Xl x (T),M(r))l m ) - 3e + £7[/(X^(T))l n \ r(fc) ] 

for every fe > 1. Letting — > oo, we have T(/c) f P-a.s. by (|2.18p 
and (|2.21|) ; therefore, 

E[f(Xl x (T))l Q \ m ]^0 

by dominated convergence and (|2.ip . Moreover, monotone convergence yields 

E[v{T,Xl x (r),M{T))l m ] ^E[v{t,XI x (t),M{t))]; 

to see this, consider separately the cases where the positive or the negative 
part of ip(r, X^ x (t), M(t)) are integrable. Hence we have shown that 

V(t, x,m + 8)> E[<p(r, X? x (t), M(t))] - 3e. 
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As e > was arbitrary, this completes the proof of (ii). 

(ii') Fix (t,x,m) G S and let e > 0. In contrast to the proof of (ii), we 
shall cover a subset of S rather than Sxl. By (|2.11|) . we may again assume 
that V n D = V. Since V > <p on V, the definition {22} of V shows that 
there exists for each (s,y,ra) G D some v s,y,n G U(s,y,n) satisfying 

F(s,y-v s >y> n )><p(s,y,n)-e. (2.28) 

Given (s, y, n) G T>, the semicontinuity assumptions (|2.16p and a variant 
of Lemma 12.11 (including the time variable) imply that there exists a set 
B(s, y, n) C S of the form 

B(s, y, n) = ((s -r,s]D [0, T}) x B r (y, n), 

where r > and B r (y, n) is an open ball in S x M + , such that 

v s,y,n G ^( s ' 5 y' )n ' + 5) ^ 

ip(s',y',n') < ip(s,y,n)+£ > for all (s', y', n') G 5(s, y, n) fl P. 
F(s / ,y / ;^ n ) > F(s,y;i/ S ^ n ) -e J 

(2.29) 

Note that we have exploited Assumption (E(0l), which forces us to use the 
half-closed interval for the time variable. As a result, B(s,y,n) is open for 
the product topology on [0, T] x S x R if S and M are given the usual topology 
and [0, T] is given the topology generated by the half-closed intervals (the 
upper limit topology). Let us denote the latter topological space by [0, T\* . 

Like the Sorgenfrey line, [0, T]* is a Lindelof space. Let D' be the canoni- 
cal projection o£T> C S = [0,T]x5x]Rto5'xR. Then V' is again <r-compact. 
It is easy to see that the product of a Lindelof space with a cr-compact space 
is again Lindelof; in particular, [0,T]* x V' is Lindelof. Moreover, since T> 
was cr-compact for the original topology on S and the topology of [0,T]* is 
finer than the original one, we still have that V C [0, T\* x V is a countable 
union of closed subsets and therefore T> is Lindelof also for the new topology. 

By the Lindelof property, the cover {B(s,y,n) n T> : (s,y,n) G S} of T> 
admits a countable subcover {B(sj,yj,rij) nD}j>i. We set Uj := v s i> y i ,n i 
and Bj := B(sj,yj,rij), then T> C Uj>iBj and 

A 1 :=B 1 , A i+ i:=B i+ i\(B 1 U---US i ), j>l 

defines a measurable partition of U j>\Bj. Since Aj C Bj, the inequali- 
ties (I2~28]> and <^29\i yield that 

F(s, y'; Uj) > ip(s' , y', n ) — 3e for all (s', y', n') G Aj n P. 

Similarly as above, we first fix k > 1, define the ^-measurable sets 

T r .= {(r,Xl x (r),M(r))GA 3 } and r(fe) := \J Fj 

l<j<k 
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and set v{k) := ( • • • ((1/ ®( r ,ri) v i) ®>{t,t 2 ) ^2) ' ' ' ®(V,r fe ) ^fc)- To check that 
E[/(^i fc) (T))|7- T ] > F(r,X^(r);^) on Tj for 1 < j < k, 

we use that t < s,- on Tj , so that we can apply (|2.10p with the stopping time 
f := r A Sj satisfying ||r||ioo < Sj and thus z/j £ W Sj C Z/umi ; c.f. (E(UJ). 
The rest of the proof is analogous to the above. □ 

Remark 2.6. The assumption on cr-compactness in Theorem I2.4f ii') was 
used only to ensure the Lindelof property of DdD for the topology introduced 
in the proof. Therefore, any other assumption ensuring this will do as well. 

Let us record a slight generalization of Theorem I2.4f ii).(ii') to the case 
of controls which are not necessarily admissible at the given point (t,x,m). 
The intuition for this result is that the dynamic programming principle holds 
as before if we use such controls for a sufficiently short time (as formalized by 
condition (|2.30p below) and then switch to admissible ones. More precisely, 
the proof also exploits the relaxation which is anyway present in (I2.15p . We 
use the notation of Theorem 12.41 

Corollary 2.7. Let the assumptions of Theorem \2.4fo i) hold true except 
that v eU t and M £ M t . m are not necessarily inlA(t,x,m) and hA^ mx {iJ), 
respectively. In addition, assume that 

(T,^(T),Jf(T))eD P-a.s. (2.30) 

Then the conclusion (|2.15p of Theorem \ 2. ) still holds true. Moreover, the 
same generalization holds true for Theorem nTW ii '). 

Proof. Let us inspect the proof of Theorem I2.4f ii). Using directly ([2.301) 
rather than appealing to ()2.1ip . the construction of the covering in Step 1 
remains unchanged and the same is true for the concatenation in Step 2. In 
Step 3, we proceed as above up to and including (|2.26p . Note that (|2,27p 
no longer holds as it used the assumption that M £ -M.tmx{ v )- However, in 

view of (I2T261) and '(T) = X^ X (T) onfl\ T(k), we still have 

E[g(X$\T))] <E[M^(T)l r[k) ] + E[g(X^(T))l n \ m ]. 

Since g(X^ x (T)) is integrable by ()2. 1|) and T(k) t ^, the latter expectation 
is bounded by 5 for large k. Moreover, T(k) £ J>, the martingale property 
of , and the fact that = M + 5 on [0, r] yield that 

E[M^(T)l m ] =£[MW(T)l rW ] =E[(M(r)+5)l m ]. 

Since E[M(t)] = m, the right hand side is dominated by m + 25 for large k. 
Together, we conclude that 

E[g(X$\T))]<m + S5; 
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i.e., v{k) £ U(t,x,m + 35) for all large k. Step 4 of the previous proof then 
applies as before (recall that f{X^ x {T)) is integrable by (|2.ip ). except that 
the changed admissibility of v{k) now results in 

V(t,x,m + 35) > E[<p(t,XZ x (t),M(t))]. 

However, since 5 > was arbitrary, this is the same as (|2.15p . The argument 
to extend Theorem I2.4f ii') is analogous. □ 

While we shall see that the relaxation 5 > in (|2.15p is harmless for the 
derivation of the Hamilton- Jacobi-Bellman equation, it is nevertheless inter- 
esting to know when the 5 can be omitted; i.e., when V is right continuous 
in m. The following sufficient condition will be used when we consider state 
constraints. 

Lemma 2.8. Let (t, x, m) € D. For each 5 > there is u s £ U(t, x, m + 5) 
such that 

F(t, x; v 5 ) > (T 1 A V(t, x,m + S)-5. 

Let do > and assume that for all < 5 < 5o there exists i> s £ U(t,x,m) 
such that 

MP{<(T)/<(T)}=0 

and such that the set 

{[f(x£(T))-f(x£(T))} + :0<5<5 } C L l {P) (2.31) 
is uniformly integrable. Then m! i— > V(t, x,m') is right continuous at m. 

Proof. Since m' \-¥ U{t, x, m') is increasing, we have that (t, x, m) G D 
implies U(t, x,m + 6) ^ for all 5 > 0. Hence exists; of course, the 
truncation at (5" 1 is necessary only if V(t,x,m + 5) = oo. Moreover, the 
monotonicity of m! i— >■ V(t, x, m') implies that the right limit V(t, x, m+) 
exists and that V(t, x, m+) > V(t,x,m); it remains to prove the opposite 
inequality. Let < 5 < 5 and set A 5 := {xf x (T) ^ X?* X (T)}, then 

V(t,x,m) > F(t, x, 9 s ) 

= F(t,x,v 5 )-E[l A s(f(x£(T))-f(x£{T)))] 

> S- 1 A V(t, x,m + 5)-8 -E[l A s (f(X&(T)) - f(X^(T))) + } . 

Letting 5 I 0, we deduce by (|2.3ip that V{t, x, m) > V(t, x, m+). □ 

Remark 2.9. The integr ability assumption (I2.3ip is clearly satisfied if / is 
bounded. In the general case, it may be useful to consider the value function 
for a truncated function / in a first step. 
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Remark 2.10. Our results can be generalized to a setting with multiple 
constraints. Given JV£N and m G WL N , let 

U(t,x,m) := {v G lit ■ G l (t,x;u) < m' for % = 1, . . . ,N], 

where G l {t,x;v) := E[g l (X^ x (T))] for some measurable function g l . In 
this case, Mtfl is defined as the set of cadlag iV-dimensional martingales 
M = {M(s), s G [t,T]} with initial value M(t) = 0, and 

M t , m :={m + M : M e M t ,o}, m G R N . 

This generalization, which has been considered in [1] within the framework 
of stochastic target problems with controlled loss, also allows to impose con- 
straints at finitely many intermediate times < T\ < T2 < • • • < T. Indeed, 
we can increase the dimension of the state process and add the components 

3 Application to State Constraints 

We consider an open set O C S := M. d and study the stochastic control 
problem under the constraint that the state process has to stay in O. Namely, 
we consider the value function 

V(t,x):= sup F(t,x;u), (t,x)eS, (3.1) 
h>eu(t,x) 

where 

U(t,x) :={ueU t : X£ x (s) G O for all s G [t,T], P-a.s.}. 

In the following discussion we assume that, for all (t,x) G S and v G lit, 

X" x has continuous paths; (3-2) 
(t,x) i—)- X^ x {r) is continuous in probability, uniformly in r; (3.3) 
U{t, x) ^ for (t, x) G [0, T] x O. (3.4) 

Explicitly, the condition fj3.3j) means that (t n ,x n ) —¥ (t,x) implies 

sup d(X? (r),XZ x (r)) -> in probability, 

re[0,T] 

where we set X^ x {r) := x for r < t, d(-,-) denotes the Euclidean metric, 
and it is implicitly assumed that v G Ut n for all n. We shall augment 
the state process so that the state constraint becomes a special case of an 
expectation constraint. To this end, we introduce the distance function 
d(x) := m£{d(x, x') : x' G S \ O} for x G S and the auxiliary process 

Y t %Js) := y A inf d(X? >x {r)), s G [t,T], y€[0,oo). (3.5) 

r€[t,s] 
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By (|3.2p . each trajectory {X^ x (r)(u>), r G [t, T]} C 5 is compact; therefore, 
it has strictly positive distance to S \ O whenever it is contained in O: 

{X^(r)(w), r G [i,T]} CO if and only if Y^ x1 {T){lS) > 0. 

We consider the augmented state process 

X t,x,y( m ) := { X t,x{-)i Y t,x,y{')) 

on the state space S x [0, oo), then 

E[g(X^ y (T))] = P{Y t » x>y (T) < 0} by setting g(x,y) := l^^y) 

for (x,y) G S 1 x [0, oo). Now the state constraint may be expressed as 
E \g( X t,x,l ( T ))l ^ and therefore 

W(£,x) =U(t,x, 1,0) and aj) = x, 1, 0); 

of course, the value 1 may be replaced by any number y > 0. Here and in the 
sequel, we use the notation from the previous section applied to the controlled 
state process Ion5x [0, oo); i.e., we tacitly replace the variable x by (x,y) 
to define the set U(t, x, y, m) of admissible controls and the associated value 
function V(t, x, y, m). 

One direction of the dynamic programming principle will be a conse- 
quence of the following counterpart of Assumption lAl 

Assumption A. For all (t,x) G S, v E U(t,x), r G T* and P-a.e. co G $7, 
there exists v w G U(t(u), X" x {t){uj)) such that 

The more difficult direction of the dynamic programming principle will 
be inferred from Theorem 12.41 under a right-continuity condition; we shall 
exemplify in the subsequent section how to verify this condition. 

Theorem 3.1. Consider (t,x) G S and a family {t u , v G U(t,x)} C T 1 ■ 
(i) Let Assumption A hold true and let (ft : S — > [— oo, oo] be a measurable 
function such that V < 4>. Then E[4>{t v , X^ x {r v ))~] < oo for all 
v G li(t, x) and 

V(t,x)< sup E[cJ)(t v , X^ x (t u ))~\ . 

(ii) Let Assumption \B 1 hold true for the state process X on S x [0, oo) and 
let (|3.2p - (|3.4p hold true. Moreover, assume that 

V(t,x, 1,0) = V(t,x, 1,0+) (3.6) 
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and that (s',x') •—>• F(s',x';v) is l.s.c. on [0, to] x O / or a ^ *o S [i, T] 
and 1/ G Ut ■ Then 

V(t,x)> sup E[4>{t v ,X^ x {t v ))} (3.7) 

i/GW(t,x) 

/or any u.s.c. function <p : S — > [— oo, oo) sttc/i that V > <fr. 

Proof, (i) We may assume that U{t,x) ^ as otherwise V(t,x) = — oo. As 
in the proof of (|2.13p . we obtain that F(t,x;v) < E[(p(t,X" x (t))] for all 
v G U(t,x). The claim follows by taking supremum over v. 

(ii) Again, we may assume that U(t,x) ^ as otherwise the right hand 
side of (|3.7p equals — oo. We set 

V := [t,T] x O x (0,oo) x {0}. 

If v G U(t,x), then g(X" x 1 (T)) = and hence the constant martingale 
M := is contained in , xl dv). Moreover, Y" xl (s) > and hence 

(s,X" x 1 (s),M(s)) € 25 for all s € [t, T], P-a.s. Furthermore, if we define 



<p(t',x',y',m') :-- 



4>{t',x'), (t',x',y',m') G V 
— oo, otherwise, 



then tp is u.s.c. on T>. To see that the third semicontinuity condition in (|2.16p 
is also satisfied, note that for any (t',x',y'), (t",x",y") G [0,io] x ^x [0, oo) 
and v G Ut , 



Ytf \x' ,y'\T) —Yiji ,x" lW " (^0 I 



<|y'-/l + 



inf d(Xi;,,(r))- inf d(XZ, x „(r)) 
re[t',r] I,IW re[t",T] 1 y 



<|y'-n"| + sup d(X^ xl (r),X^ l>xll (r)). 

re[0,T] 

Hence (|3.3|) implies that (t', a;', y') i— >■ YJf x , y /(T) is continuous in probability. 
As (— oo, 0] C R is closed, we conclude by the Portmanteau theorem that 



(*', x', 7/0 ^ P{Yl x , jy , (T) G (oo, 0] } = G(t', x', y'; v) 



IS u.s.c. 



as required. Since any open subset of a Euclidean space is cr-compact and 
since (|3.4p implies T> n D = V, we can use ()3.6[) and Theorem !2.4l fii ,N > with 
M = to obtain that 

V(t,x) = V(t,x,l,0) 
= V(t,x,l,0+) 

> ^[^,^(0,^(0,0)] 

= E[^(^,x^(0)]. 

As ^ G U(t,x) was arbitrary, the result follows. □ 
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Remark 3.2. Similarly as in Theorem 12, 4f ii). there is also a version of The- 
orem [3J](ii) for stopping times taking countably many values. In this case, 
Assumption [B] replaces Assumption [Bll all conditions on the time variable 
are superfluous, and one can consider a general separable metric space S. 

4 Application to Controlled Diffusions 

In this section, we show how the weak formulation of the dynamic program- 
ming principle applies in the context of controlled Brownian stochastic dif- 
ferential equations and how it allows to derive the Hamilton- Jacobi-Bellman 
PDEs for the value functions associated to optimal control problems with 
expectation or state constraints. As the main purpose of this section is to il- 
lustrate the use of Theorems 12.41 and 13 . 1 1 we shall choose a fairly simple setup 
allowing to explain the main points without too many distractions. Given 
the generality of those theorems, extensions such as singular control, mixed 
control/stopping problems, etc. do not present any particular difficulty. 

4.1 Setup for Controlled Diffusions 

From now on, we take S = M. d and let = C([0,T];M. d ) be the space 
of continuous paths, P the Wiener measure on f2, and W the canonical 
process Wt(oS) = uif. Let F = (J~t)t£[Q,T] ^> e the augmentation of the filtration 
generated by W; without loss of generality, T = Tt- For t € [0,T], the 
auxiliary filtration F* = {J~l) S £[o,T] i s chosen to be the augmentation of 
a(W r — Wt, t < r < s); in particular, F* is independent of Tt- 

Consider a closed set U C M. d and let Li be the set of all [/-valued pre- 
dictable processes v satisfying E[J^ \ut\ 2 dt] < oo. Then we set 

Ut = \ v €zU : v is F*-pr edict able}. 

This choice will be convenient to verify Assumption B'. We remark that the 
restriction to F'-predictable controls entails no loss of generality, in the sense 
that the alternative choice Ut =li would result in the same value function. 
Indeed, this follows from a well known randomization argument (see, e.g., 
O Remark 5.2]). 

Let M. d denote the set of d x d matrices. Given two Lipschitz continuous 
functions 

H : R d x U R d , a : R d x U -»■ M d 

and (t, x, v) £ [0, T] x IR d x U, we define X^ x (-) as the unique strong solution 
of the stochastic differential equation (SDE) 

X(s) = x+ n(X(r),i/ r )dr+ a(X(r),v r ) dW r , t<s<T, (4.1) 
Jt Jt 
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where we set X^ x {r) = x for r < t. As X^ X (T) is square integrable for any 
v €U, (12.11) is satisfied whenever 



/ and g have quadratic growth, (4-2) 
which we assume from now on. In addition, we also impose that 



/ is l.s.c. and / has subquadratic growth, 
g is u.s.c. and g + has subquadratic growth, 



(4.3) 



where h : ~R d — > M is said to have subquadratic growth if h(x)/\x\ 2 as 
| a; | — > oo. This will be used to obtain the semicontinuity properties (|2.16p . 

Furthermore, we take Mt,o to be the family of all cadlag martingales 
which start at and are adapted to F . By the independence of the incre- 
ments of W, we see that M G Mtfl is then also a martingale in the filtration 
F* and that M r = for r < t. For v G U u we have X" X (T) G L 2 (J%,P) 
and hence (J23D is satisfied with M?[x](-) = E[Xl x (T)\P;]. It will be useful 
to express the martingales as stochastic integrals. Let At denote the set of 
Revalued F*-predictable processes a such that \at\ 2 dt < oo P-a.s. and 
such that 

is a martingale ( T denotes transposition). Then the Brownian representation 
theorem yields that 

M t ,o = {M£ : a G At] ■ 
In the following, we also use the notation M® m := m + M t Q . 

Lemma 4.1. In the above setup, Assumptions A, A, B, B' are satisfied and 
F and G satisfy (l2~T6|) . 

Proof. Assumption (BO') is immediate from the definition of Uf We define 
the concatenation of controls by (|2.12p . 

The validity of Assumptions A, A and (Bl') follows from the uniqueness 
and the flow property of f|4. 1 1> : in particular, the control v w in Assumption A 
can be defined by i/ u (u') := v((jj® T u'), oj' G f2, where the concatenated path 
uj <S> T U)' is given by 

(w ® r uj') r := w r l[ 0jT (a,)] (r) + (u' r - u' T (u) + u T (u)) l( r H,r] (r). 

While we refer to [31 Proposition 5.4] for a detailed proof, we emphasize 
that the choice of IA S is crucial for the validity of ()2. lOj) : in the notation of 
Assumption B', (|2.10p essentially requires that v be independent of T T . 

Let t,T,u,u be as in Assumption B', we show that (B2') holds. Let M 
be a cadlag version of 

M(r) := E[g(Xl x (T))\T r ], r G [0,T], where v := i/1 [0jT] + l^ T] u. 
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By the same argument as in [3l Proposition 5.4], we deduce from the unique- 
ness and the flow property of (|4.ip and the fact that v is independent of J- T 
that 

E\ g {Xt x (T))\F r ] = E[g(X^ xlx{T) (T))\F r ] = M*[X» x {r)]{r) on [r,T]. 

Hence M = M*[X% >x {t)\ on [r,T]. The last assertion of (B2') is clear by the 
definition of Mtfi- As already mentioned in Remark 12.31 Assumption (B3') 
follows from Assumption A and Assumption B follows from Assumption B'. 

Next, we check that F satisfies (|2.16p ; i.e., that F is l.s.c. For fixed u Eli, 
(t,x) i— > X^ X (T) is L 2 -continuous. Hence the semicontinuity from (|4.3|) 
and Fatou's lemma yield that (t,x) i— > E[f(X^ x (T)) + ] is l.s.c. By the sub- 
quadratic growth from (|4.3p . we have that {f(X^ x (T))~ : (t,x) G B} is 
uniformly integrable whenever B C S is bounded, hence the semicontinuity 
of / also yields that (t,x) h4 E[f(X^ x (T))~] is u.s.c. As a result, F is l.s.c. 
The same arguments show that G also satisfies (|2.16p . □ 

4.2 PDE for Expectation Constraints 

In this section, we show how to deduce the Hamilton- Jacobi-Bellman PDE 
for the optimal control problem (|2 . 2[) with expectation constraint from the 
weak dynamic programming principle stated in Theorem 12.41 Given a suit- 
ably differentiable function ip(t, x) on [0, T] x R d , we shall denote by dt<p 
its derivative with respect to t and by Dip and D 2 ip the Jacobian and the 
Hessian matrix with respect to x, respectively. 

In the context of the setup introduced in the preceding Section 14.11 the 
Hamilton-Jacobi-Bellman operator is given by 

H(x,p,Q):= inf ( - L u ' a (x,p,Q)), (x,p, Q) G R d x R d+1 x M d+1 , 

(u,a)eUxR d 

where 

L u ' a (x,p,Q) := fi x ,M{x,u) T p + ^Tr[ax,M°'x,M(x,u,a)Q}, (u,a) G U xR d 
is the Dynkin operator with coefficients 

(J>x,m{x,u) := I o J ax,M[x,u,a) := I y J. 

Since the set U x ]R d is unbounded, may be discontinuous and viscos- 
ity solution properties need to be stated in terms of the upper and lower 
semicontinuous envelopes of H, 

H*(x,p,Q):= limsup H(x',p',Q'), 

(x',p',Q')-^(x,p,Q) 

H*(x,p,Q):= liminf H(x',p',Q'). 

(x',p',Q')^(x,p,Q) 
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The value function V defined in (j2.2j) may also be discontinuous and so we 
introduce 

V*(t, x, m) := limsup V(t',x,m), 

(t , x f , m ) — > (£, a;, m) 
(t',a/, m') g int D 

x,m) := liminf V(t',x,m). 

(t , X , m ) — > (£, a:, m) 
(*', x', m') g int D 

Here int D denotes the parabolic interior; i.e, the interior of D \ \t = T} in 
S, where {t = T} := {(t,x,m) € S : t = T}. Moreover, we shall denote by 
D the closure of D. The main result of this subsection is the following PDE. 

Theorem 4.2. Assume that V is locally bounded on intD. 

(i) The function V* is a viscosity subsolution on D \ {t = T} of 

-dt<p + H m (-,D<p,lP(p) < 0. 

(ii) The function V* is a viscosity supersolution on intD of 

-d t ip + H*(;D<p,D 2 <p) > 0. 

We refer to [5] for the various equivalent definitions of viscosity super- 
and subsolutions. We merely mention that "subsolution on A" means that 
the subsolution property is satisfied at points of A which are local maxima of 
V* — if on A, where ip is a test function, and analogously for the supersolution. 

We shall not discuss in this generality the boundary condition and the 
validity of a comparison principle. In the subsequent section, these will be 
studied in some detail for the case of state constraints. We also refer to [1] 
for the study of the boundary conditions in a similar framework. 

Remark 4.3. We observe that the domain of the PDE in Theorem 14.21 is 
not given a priori; it is itself characterized by a control problem: if we define 

v(t,x) := inf E[g(Xf (T))], (t,x) G S, (4.4) 
v&At 

then 

int D = { (t, x, m) € S : m > v*(t,x), t < T}, 

where v* is the upper semicontinuous envelope of v on [0, T) x R d . In partic- 
ular, intD ^ since v is locally bounded from above. In fact, in the present 
setup, we also have 

intD = {(t,x,m) eS:m> v(t,x), t < T}. (4.5) 

Indeed, a well known randomization argument (e.g., j3j Remark 5.2]) yields 
that v(t,x) = ini ueU E[g(X£ x (T))] for all (t,x) 6 S; i.e., the set U t in flHD 
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can be replaced by U. Therefore, v inherits the upper semicontinuity of G 
(c.f. Lemma l4.ip and we have v = v* . Using (|4.5p . we obtain that 

{(i, x, m) £ S : m > v(t, cc)} C x,m) € S : m > «*(£, cc)} = intD, 

where i>* is the lower semicontinuous envelope of v on [0, T] x R d , and hence 

D C intD. (4.6) 

If furthermore v is continuous and the infimum in (|4.4p is attained for all 
(t,x), then the converse inclusion is also satisfied. In applications, it may be 
desirable to have continuity of v so that int D and D are described directly 
by v (rather than a semicontinuous envelope). To analyze the continuity of 
v, one can study the comparison principle for the Hamilton- Jacobi-Bcllman 
equation associated with the control problem (|4.4p . We refer to [U Sec- 
tions 5 and 6] for the explicit computation of v in an example from Mathe- 
matical Finance. 

The rest of this subsection is devoted to the proof of Theorem 14.21 We 
first state a version of Theorem 12.41 which is suitable to derive the PDE. 

Lemma 4.4. (i) Let B be an open neighborhood of a point (t,x,m) € D 
such that V(t, x, m) < oo and let if : B — > R be a continuous function such 
that V < ip on B. For all e > there exist (v, a) G Ut X At such that 

V(t,x,m) < E[<p(r,^(r),M^ m (T))] +e and M t a m (T) > g(X^(T)), 

where t is the first exit time of (s, X^ x {s), M" m (s)) s >j from B. 

(ii) Let B be an open neighborhood of a point (t,x,m) € intD such that 
B C D and let (v,at) £ W( X At- For any continuous function (p : B — > R 
satisfying V > ip on B and for any e > 0, 

V(t,x,m + e) > Ey(T,Xl x (r),M t a m (r))], (4.7) 

where t is the first exit time of (s, X^ x {s), M" m (s)) s >t from B. 

Proof. In view of Lemmata 12.21 and 14.11 part (i) is immediate from Theo- 
rem E3ti). 

For part (ii) we use the extension of Theorem 12.4( 11') as stated in Corol- 
laryOwith V := B. Note that (r, X£ x (t), M£ x (t)) e5CD;in particular, 
T> n D = T> is closed and hence cr-compact. □ 

We can now deduce the PDE for V from the dynamic programming 
principle in the form of Lemma 14.41 Although the arguments are the usual 
ones, we shall indicate the proof, in particular to show that the relaxation 
"m + e" in (HTJ) does not affect the PDE. 
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Proof of Theorem (i) We first prove the subsolution property. Let ip 
be a C 1,2 -function and let (to,£o, m o) £ D be such that to £ (O,? 1 ) and 
(to, xo, mo) is a maximum point of V* — ip satisfying 

(V* -ip)(t ,x ,m ) = 0. (4.8) 

Assume for contradiction that 

(-%? + H :¥ (- 1 D(p,D 2 ip))(t ,x ,m ) > 0. 



Since D = intD by (|4.6p . there exists a bounded open neighborhood BcS 
of (to,iCO) m o) such that 

-d t Cp-L u ' a {-,D(p,D 2 (p) > onSnmtD, for all («, a) € U x K d , (4.9) 

where 

<£(s, y, n) := (p(t 0) x , m ) + (|s - to| 2 + |?/ - x | 4 + |n - m | 4 ) . 
Moreover, we have 

T] := min((^ - <p) > 0. (4-10) 

Given e > 0, let (t e ,x £ , m £ ) G B n int D be such that 

F(t e , x e , m e ) > 7*(t , ^o, m ) - e. (4.11) 

Consider arbitrary (v,a) ZLlAtX At such that M t " ms (T) > g(X t ^ j. £ (T)) and 
let r be the first exit time of (s,X^ Xe (s),Mj* m£ (s)) s >t from B. We recall 
from Remark I2,3f ii) that (s,X" x (s),M" m (s)) s >t remains in D on [£,T], 



and hence also in intD by (|4.6|l . Now, it follows from Ito's formula and (|4.9p 
that 

^(t e ,x e ,m £ ) > £[£(T,X t %>),M£ )fn- (r))]. 
For (t e ,x £ ,m e ) close enough to (to,xo,mo), this implies that 

^(to,x ,mo)>^[v(r,^ )a!s (r),Afg >me (r))] - o(l) as e -> 0, 

which, by (|Q]> . (|4TTU|) and (pTTTj) . leads to 

7(t B) z e ,m E )>£[^(T,^ iIt (r),J^ lBI ,(r))] + 77-0(1). 

This contradicts Lemma l4.4n ) for e > small enough. 

(ii) We now prove the supersolution property. Let (p be a C 1,2 -function 
and let (to,Xo,mo) € intD be such that (to,Xo,mo) is a minimum point of 
V* — </? satisfying 

(K -^)(t ,xo,m ) = 0. (4.12) 
Assume for contradiction that 

(-d t p + H*(; Dip, D 2 ip)) (t , so, ^o) < 0. 
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Then there exist (u, a) G U x M d and a bounded open neighborhood B of 
(io,£(b m o) such that 23 C intD and 

-d t tp-L^ & (-,D<p,D 2 <p) < onB, (4.13) 

where 

<p(s,y,n) := p(t ,x ,m ) - (|s - t | 2 + 1 2/ - | 4 + |n - m | 4 ). 
Note that 

r] := min(<£ — > 0. (4-14) 
8B 

Given e > 0, let (t e , x e , m e ) 6 B be such that 

F(t e ,x e ,m e + e) < K(io,a;o,mo) + e. (4.15) 

Viewing (u, a) as a constant control, it follows from Ito's formula and (|4.13|) 
that 

(p{t E ,x £ ,m e ) < E[tp(T,Xl Xe (T),M t l me (r))], 

where r is the first exit time of (s, Xs .(s), me (s)) s >( from B. For 
(t e ,x e ,m e ) close enough to (to, ^ch m o)> this implies that 

£(t ,x ,m ) < E[^(T,Xl >Xs (T),Ml )ms (r))]+o(l) as e -> 0, 

which, by P~T2|) . (|4"T4"1) and (j4~T5|) . leads to 

F(i £ ,x e ,m £ + £ ) < ^(r^^M*^))] -77 + 0(1). 

For e > small enough, this yields a contradiction to Lemma l4.4f ii). □ 

4.3 PDE for State Constraints 

In this section, we discuss the Hamilton- Jacobi-Bellman PDE for the state 
constraint problem (cf. Section [3]) in the case where the state process is given 
by a controlled SDE as introduced in Section 14.11 and required to stay in an 
open set O C M. d . Note that in this setup, the continuity conditions (|3.2p 
and (|3.3p are satisfied. 

We shall derive the PDE via Theorem 13.11 The basic idea to guaran- 
tee its condition (|3.6p about right continuity in the constraint level runs as 
follows. Consider a control v 6 lit such that X\ x leaves O with at most 
small probability S > 0. Then we shall construct a control v s , satisfying the 
state constraint, by switching to some admissible control u shortly before 
X^ x exits O. As a result, v s coincides with v on a set of large probability 
and therefore the reward is similar. Along the lines of Lemma 12.81 we shall 
then obtain the desired right continuity (cf. Lemma 14.71 below) . 

To make this work, we clearly need to have U(t, x) ^ for all (t, x) in 
[0, T] x O, which is anyway necessary for the value function V from (|3.ip 
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to be finite. However, we need a slightly stronger condition; namely, that 
we can switch to an admissible control in a measurable way. A particularly 
simple condition ensuring this, is the existence of an admissible feedback 
control: 

Assumption C. There exists a Lipschitz continuous mapping u : O —> U 
such that, for all (t, x) G [0, T] x O, the solution Xt x of 

X{s) = x+f ti{X(r),u(X(r)))dr+ f a (X (r) , u(X (r))) dW r , s G [t,T] 
Jt Jt 

(4.16) 

satisfies Xt x (s) G O for all s G [t,T], P-a.s. 

If, e.g., /j,(-,uq) = and cr(-, uq) = for some no G U, then Assumption C 
is clearly satisfied for u = uq. Or, under an additional smoothness condition, 
Xt x will stay in O if the Lipschitz function u satisfies 

\na\(-,u)=0 and ^?t, t u + — Trace[Dn crcr T ]^ (•, u) > on dO, 

where n denotes the inner normal to dO; see also Proposition 3.1] and 
[3 Lemma III.4.3]. 

The following is a simple condition guaranteeing the uniform integrability 
required in (|2.3ip . 

Assumption D. Either / is bounded or the coefficients fi(x,u) and a(x,u) 
in the SDE (|4.ip have linear growth in x, uniformly in u. 

This assumption holds in particular if the control domain U is bounded. 

Remark 4.5. Assumption O implies that V is locally bounded from below 
and Assumption [D] implies that V is locally bounded from above. 

Next, we introduce the notation for the PDE related to the value function 
V from (|3.ip . The associated Hamilton- Jacobi-Bellman operator is given by 

H(x,p,Q) := inf ( - L u (x,p,Q)) , (x,p, Q) G R d x R d x M d , (4.17) 
where the Dynkin operator is defined by 

L u (x,p,Q) := n(x,u) T p + - Ti[aa T (x,u)Q], u eU. 
Similarly as above, we introduce the semicontinuous envelopes 
H*(x,p,Q):= limsup H(x',p',Q'), 

(x',p' : Q') -> Q) 
(x',p',Q') e O X M d x M d 

H*{x,p,Q):= liminf H(x',p',Q') 

(x',p', Q') -> (at, p, Q) 
(x',p',Q') e O X M d x M d 
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and 

V*(t,x) := limsup V(t',x'), 

(£' , x') — > (t, x) 

(t' ,x') e [o, T) x o 
V*(t,x):= liminf V(i!,rf). 

{t' , x') — f (t, x) 
(*', m') S [0, T) x O 

We can now state the Hamilton- Jacobi-Bellman PDE; the boundary con- 
dition is discussed in Proposition 14. 1 ll below. 

Theorem 4.6. Assume that V is locally bounded on [0,T) x O. 
(i) The function V* is a viscosity subsolution on [0, T) x O of 

-d t ip + H*(;D(p,D 2 ip) < 0. (4.18) 

(ii) Under Assumptions^ and [3 the function V* is a viscosity supersolu- 
tion on [0, T) x O of 

-d t <p + H*(;Dip,D 2 tp) > 0. 

The proof is given below, after some auxiliary results. We first verify 
the right-continuity condition fj3.6j) for the value function V(t, x, y, m) intro- 
duced in Section [3l 

Lemma 4.7. Let Assumptions [Cl and\fi\ hold true. Then V(t, x, 1, 0+) = 
V{t, x, 1,0) for all (t, x) € [0, T] x O. 

Proof For 6 > 0, let v s € U(t,x,S) be as in Lemma 12.81 Then the process 
Ytxy defined in (|3.5p satisfies Y" x i(T) > outside of a set of measure at 
most 5. It follows that we can find £5 £ (0, 1 A d(x)) such that the set 
A s := {Yf xl (T) < e s } satisfies P[A S ] < 25. Let t s denote the first time 

when Y" x 1 reaches the level £$ and set 

9 s := u'l^+l^s^uiX 6 ), 

where X s is the solution of (|4.16p on [t^T] with initial condition given by 
X s (t s ) = X^ x (t s ). Since the paths of Y" x 1 are nonincreasing, we have 

hmPpTg(T) ^ X&(T)] < ljmP[r 5 <T} = limP[/] = 0. 

Next, we check that {f(X^ x (T)), v € U} is uniformly integrable. This is 
trivial if / is bounded. Otherwise, Assumption [D] yields that the coefficients 
fi(x, u) and <j(x, u) have uniformly linear growth in x, and of course they are 
uniformly Lipschitz in x as they are jointly Lipschitz. Thus {X^ X (T), v € U} 
is bounded in LP for any finite p and the uniform integrability follows from the 
quadratic growth assumption (|4.2p on /. It remains to apply Lemma [2.81 □ 
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Lemma 4.8. Let Assumption^ hold true and let B be an open neighborhood 
of (t, x) € [0, T] x O. For all u &Ut there exists u £Ut such that 

where r is the first exit time of (s, X^ x (s)) s >t from B. 

Proof. Let X T> x» ( T ) be the solution of ()4.16p on [r, T] with the (square 
integrable) initial condition X^ x {t) at time r. Then the claim holds true for 
v ■= fl[t, T ] + 1 (t,T}u(X TiX , x(t) ). □ 

We have the following counterpart of Lemma 14.41 

Lemma 4.9. (i) Let B C [0, T] x IR^ be an open neighborhood of a point 
(t, x) € [0, T]xO such that V(t, x) is finite and let <p : B —?■ M be a continuous 
function such that V < (p on B. For all e > there exists v € U(t,x) such 
that 

V(t,x)<E[tp(T,X^(r))] +e, 

where r is the first exit time of (s, X^ x {s)) s >t from B. 

(ii) Let Assumptions [Cl and [21 hold true and let B C [0, T] x O be an 
open neighborhood of (t,x). For any v £ lit and any continuous function 
<f : B — )• M satisfying V > (p on B, 

V(t,x)>E[<p(T,X» x (r))], 

where r is the first exit time of (s, X^ x {s)) s >t from B. 

Proof. In view of Lemma 14.11 part (i) is immediate from Theorem 13.1( 1). 
Part (ii) follows from Theorem I3.1f ii) via Lemmata 14.71 and 14.81 . □ 

Proof of Theorem \4-6\ The result follows from Lemma 14.91 by the arguments 
in the proof of Theorem 14.21 □ 

4.3.1 Boundary Condition and Uniqueness 

In this section, we discuss the boundary condition and the uniqueness for 
the PDE in Theorem 14.61 We shall work under a slightly stronger condition 
on our setup. 

Assumption D'. The coefficients fi(x,u) and a(x,u) in the SDE (|4.ip have 
linear growth in x, uniformly in u. 

We also introduce the following regularity condition, which will be used 
to prove the comparison theorem. 

Definition 4.10. Consider a set O C R d and a function w : [0, T] x O -> E. 
Then w is of class 1Z((D) if the following hold for any (t, x) € [0, T) x dO: 
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(i) There exist r > 0, an open neighborhood B of x in R and a function 
£ : R + -> M d such that 

liminf e _1 |£(e)| < oo and (4.19) 
y + £(e) + o(e) G for all y G n B and e € (0, r). (4.20) 

(ii) There exists a function A : — > such that 

lim A(e) = and (4.21) 
kmw(t + \(e),x + ((e)) = w(t,x). (4.22) 

By (|4.20p we mean that if tp : M. + — > M. d is any function of class o(e), 
then there exists r > such that y + + ip{e) G for all e G (0,r). 
Note that (i) is a condition on the boundary of dO; it can be seen as a 
variant of the interior cone condition where the cone is replaced by a more 
general shape. Condition (ii) essentially states that w is continuous along 
at least one curve approaching the boundary point through this shape. In 
Proposition 14. 12l below, we indicate a sufficient condition for V* to be of class 
1Z(0), which is stated directly in terms of the given primitives. We shall see 
in its proof that Definition 14.101 is well adapted to the problem at hand (see 
also Remark IA.4I below) . Before that, let us state the uniqueness result. 

Proposition 4.11. Let f be continuous and let Assumptions^ and \D] hold 
true. Then V has quadratic growth and the boundary condition is attained 
in the sense that 

V*(T,-)<f and K(T,-)>/ on O. 

Assume in addition that V* is of class 1Z((D). Then 

(i) V is continuous on [0, T] x O and admits a continuous extension to 
[0,T]xO, 

(ii) V is the unique (discontinuous) viscosity solution of the state constraint 
problem 

-d t <p + H(;D<p,D 2 <p)=0, <p(T,-)=f 

in the class of functions having polynomial growth and having a lower 
semicontinuous envelope of class 1Z{0). 

Proof. Recalling that / has quadratic growth (|4.2p . it follows from stan- 
dard estimates for the SDE (|4.ip under Assumptions O and [D] that V has 
quadratic growth and satisfies the boundary condition. 

Assumption [D] implies Assumption [D] and hence Theorem I4.6I yields 
that V* and V* are sub- and supersolutions, respectively. Moreover, As- 
sumption [D] implies that H is continuous and satisfies Assumption [E] in 
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Appendix [Aj see Lemma I A, 31 below for the proof. If V* is of class 1Z{0), 
the comparison principle (Theorem I A. II and the subsequent remark, cf. Ap- 
pendix|A]) yields that V* = V* on [0, T] x O; in particular, (i) holds. Part (ii) 
also follows from the comparison result. □ 

We conclude this section with a sufficient condition ensuring that V* 
is of class 7Z((D); the idea is that the volatility should degenerate so that 
the state process can be pushed away from the boundary. We remark that 
conditions in a similar spirit exist in the previous literature (e.g., HOj): 
cf. Remark IA.4I below. 

Proposition 4.12. Assume that V* is finite-valued on [0, T] x O and that 
O, jjL and a satisfy the following conditions: 

(i) There exists a C 1 -function 5, defined on a neighborhood of O C M. d , 
such that D5 is locally Lipschitz continuous and 

5>0 onO, 5 = on dO, 5 < outside O. 

(ii) There exists a locally Lipschitz continuous mapping u : M. d — > U such 
that for all x £ O there exist an open neighborhood B of x and i > 
satisfying 

[i(z,u(z)) T D5(y) > i and a(y,u(y)) = for all y G B(~)0 and z £ B. 

(4.23) 

Then V* is of class 1Z((D). 

Proof. Fix (t, x) € [0, T) x dO and let 5, i and B and be as above; we may 
assume that B is bounded. Consider y G O n B. Since x' h-> fi(x',u(x')) is 
locally Lipschitz, the ordinary differential equation 

%( s ) = V + / fJ.(x(r),u(x(r))) dr 
Jo 

has a unique solution x y on some interval [0,T y ). Then ()4.23p ensures that 
x y (s) £ O for s £ (0, e], for e > small enough, for all y £ Of)B. (4.24) 
For e £ [0, T x ), we set 

1(e) := x x (e) - x, A(e) := e. 

Then (|4,2ip is clearly satisfied. Using the continuity of \i and u, we see that 

£ _1 |%)| n{x,u{x)), 

which implies (|4.19p . Moreover, using that /i and u are locally Lipschitz, we 
find that 

%x(r) - x y {r) = x - y + 0(r). 
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Together with (|4,23p . (j4.24j) and the local Lipschitz continuity of fj,, u and 
DS, this implies that for y G O sufficiently close to x and e > small enough, 

S(y + £(s)+o(s)) 
= 5{y - x + x x {e)) + o(e) 

= d(y)+ / fi(x x (r),u(x x (r))) T D5(y - x + x x (r)) dr + o(e) 

JO 

= <%) + / M x - y + %(r), u(x — y + x 3/ (r))) T D5(± J ,(r)) dr + 0(e 2 ) + o(e) 
Jo 

> si + o(e), 

which is strictly positive for e > small enough. This implies (|4,20p . 

Consider (s,y) £ [0, T) x close to (t,x). For e > small enough, we 
can find v e 6 W(s + A(e),x 3/ (e)) such that 

E[f{xf +x{£ ^ y{£) (T))} > V{s + \(e),x y (e))-e. 

Recall the degeneracy condition in (|4.23p . Setting 

v e := u(x y )l[^ s+X ( £ )] + l( s+A ( £ ) jT ]Z/ e , 

we obtain that 

V(s,y)>E[f(X** y (T))] 

= E[f(Xf +x{£lXy{£) (T))] 
> V(s + X(e),x y (e)) - e. 

Recalling that x y {e) — > x x {e) as y — > x, this leads to 

K(i,x) > t4(* + A(e),x x (e)) -e = K(t + A(e),a? + £(e)) -e 

for e > small enough, which implies 

V*(t,x) > hmsupVJt + X{e),x + £(e)) 

e^O 

> lirrnnf V* (t + A(e), x + l{e)) 

> %{t,x). 

Hence lim e ^ % (t + X(e),x + £(e)) = K(i,a;); i.e., (I4T22D holds for K. □ 
4.3.2 On Closed State Constraints 

Recall that the value function V considered above corresponds to the con- 
straint that the state processes remains in the open set O. We can similarly 
consider the closed constraint; i.e., 

V(t,x) := sup {E[f(Xl x (T))\ : v eU t , X y tx {s) € O for all s € [t,T], P-a.s.} 
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The arguments used above for V do not apply to V. Indeed, for the closed 
set, the constraint function G in Section [3] would not be u.s.c. and hence the 
derivation of Theorem 13.11 fails : note that the upper semicontinuity is essen- 
tial for the covering argument in the proof of Theorem 12, 4f ii).(ii'). Moreover, 
the switching argument in the proof of Lemma [4.81 cannot be imitated since, 
given that the state process X" x hits the boundary dO, it is not possible to 
know which trajectories of the state will actually exit O. 

However, we shall see that, if a comparison principle holds, then the 
dynamic programming principle for the open constraint O is enough to fully 
characterize the value function V associated to O. More precisely, we shall 
apply the PDE for V and its comparison principle to deduce that V = V 
under certain conditions. Of course, the basic observation that this equality 
holds under suitable conditions is not new; see, e.g., |10| . We shall use the 
following assumption. 

Assumption C Assumption ICl holds with u defined on O. 

Corollary 4.13. Let f be continuous, let Assumptions\U} and\D}hold true 
and assume that V* is of class 1Z{0). Then V = V on [0,T] x O. 

We recall that V admits a (unique) continuous extension to [0, T] x O 
under the stated conditions, so the assertion makes sense. 

Proof. The easier direction of the dynamic programming principle for V can 
be obtained as above, so the result of Lemma f4.9n ) still holds. It then follows 
by the same arguments as in the proof of Theorem 14.61 and Proposition 14.111 
that the upper semicontinuous envelope V is a viscosity subsolution of (|4.18p 
satisfying V (T,-) < f on O. As in the proof of Proposition 14.111 we can 
apply the comparison principle of Theorem IA.1I to deduce that V < V* on 
[0, T] x O. On the other hand, we clearly have V < V on [0, T] x O by the 
definitions of these value functions. Therefore, we have 

v*<v*<v* <V* = V*, 

where V* denotes the lower semicontinuous envelope of V and the last equal- 
ity is due to Proposition ^. Ill It follows that all these functions coincide. □ 

A Comparison for State Constraint Problems 

In this appendix we provide, by adapting the usual techniques, a fairly gen- 
eral comparison theorem for state constraint problems which is suitable for 
the applications in Proposition 14. 1 ll and Corollary 14.131 

In the following, H denotes a continuous mapping from R d x R d x M d 
to R which is nonincreasing in its third variable, O is a given open subset of 
R rf , and p > is a fixed constant. We consider the equation 

P ip-d t ip + H(;D<p,D 2 ip) = (A.l) 
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and the following condition on H. 
Assumption E. There exists a > such that 
liminf (ft(y,g,y) -H{x,p,X^)) 

< a(\x-y\(l + \q\ + n 2 \x - y\) + (1 + \x\)\p - q\ + (1 + \x\ 2 )\Q\ 



for all (x,y) € O with |x - y\ < 1 and for all (p,q,Q) £ R d x R d x M 2a! , 
(X 7 ?, y% >0 C M d x M d and n > 1 such that 

X* \ _ . 2 



where 



, < A n + r/A^ for all 77 > 0, 



Theorem A.l. Let Assumption lEl /to/a 1 irae. Lei ioi 6e an ti.s.c. viscosity 



subsolution on O and let W2 be an l.s.c. viscosity super solution on O of (jA.ip . 
If wi and u>2 have polynomial growth on O and if W2 is of class TZ(0), then 

u>2 > w\ on {T} x O implies W2 > w\ on [0, T] x O. 

Remark A. 2. Our result also applies to the equation 

-d t <p + H(;Dcp,D 2 <p) =0, (A.2) 

provided that rl is homogeneous of degree one with respect to its second and 
third argument, as it is the case for the Hamilton- Jacobi-Bellman operators 
in the body of this paper. Indeed, w\ is then a subsolution of (|A.2p if and 
only if (t,x) 1— > e pt wi(t,x) is a subsolution of (jA.ip . and similarly for the 
supersolution. Further extensions could also be considered but are beyond 
the scope of this paper. 

Proof of Theorem \A.l\ Assume that W2 > W\ on {T} x O. Let p > 1 and 
C > Obe such that w x {t,x) -w 2 {t,x) < C(l + |x| p ) for all (t,x) € [0,T]xO. 
Assume for contradiction that sup(u;i — W2) > 0, then we can find l > and 
(t ,x ) £ [0,T] xO such that 

C := {wi — 2(p — W2)(to,xo) = max_(«Ji — 2(f) — W2) > 0, (A. 3) 

[0,T]xO 

where 

(p{t,x) := Le~ Kt {l + \x\ 2p ). 
Here n > is a fixed constant which is large enough to ensure that 

m(t,x) := (A.4) 
- pcp{t,x) + d t (j){t,x) + a{{\ + \x\)\D<f>{t,x)\ + (1 + \x\ 2 )\D 2 (t){t,x)\) 
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is nonpositive on [0, T] x ]R d , where a is the constant from Assumption [El 
Note that by the assumption that w 2 > W\ on {T} x O, we must have 

(to, s ) e [0,T) xo, 

Case 1: x G <90. For all n > 1, there exist (t n , x™, s n , y n ) G ([0, T] x O) 2 
satisfying 

$ n (t n ,x n ,s n ,y n ) = max $ n , (A.5) 

([0,T]xO) 2 

where 

$ n (t, x, s, y) := Wl (t, x) - w 2 (s, y) - @ n (t, x, s, y) 

and 

6 n (t, x, s, y) := \n 2 {\t + A(n _1 ) - s| 2 + e\x + ^(n" 1 ) - y| 2 ) 

+ |t - to I 2 + |x - x | 4 + 0(t, x) + 0(s, y), 

with ^ and A given for xo as in the statement of the Definition I4.10[ and 
e > 0. Note that ()A.5j) . the assumption that w 2 satisfies (|4.22p . and (|A.3P 
imply that 

$ n (t n , x n , s n , y n ) > <£ n (t , x , t + A^" 1 ), x + l{n~ 1 )) 
= (wi -2<t>- w 2 )(t ,x ) + o(l) 
= ( + o(l) as n -> oo. (A. 6) 

Recalling the growth condition on w\ , w 2 and the definition of eft, it fol- 
lows that, after passing to a subsequence, (t n , x n , s n , y n ) converges to some 
(t 00 ^ 00 ,* 00 ^ 00 ) G ([0,T] xO) 2 . We then have 

C = (wi - 24>-w 2 ){t ,x ) 

= max_(wi — 2<J) — w 2 ) 
[0,T]xO 

> ( Wl - 20 - ^(i 00 , x°°) - \t°° - t | 2 - |x°° - x | 4 
-\ims\\ V -n 2 {\t n + \{n~ l ) - s n \ 2 + e\x n + l{n~ l ) - y n \ 2 ) 

n—^oo 2 

> liminf$ n (f\x n ,s n ,y n ) 

n— )-oo 

>c, 

where (|A.6P was used in the last step. After passing to a subsequence, we 
deduce that 

(t n ,x n ,s n ,y n )^(t ,x ,t ,x ), (A.7) 
Wl (t n , x n ) - w 2 (s n , y n ) -> (wi - w 2 )(t , x ), (A.8) 
s ™ = t n + \( n - 1 ) + o(n- 1 ), y n = x n + £(n- 1 ) + o(n- 1 ). (A.9) 

Since (t ,x ) G [0,T) x (90, it follows from (l4~20l) . (IQTjl and (jA~9l) that 
(s n ,y n ) G [0, T) x O for n large enough. 
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2 + 2 — 

Let Vq w\ and Vq w 2 be the "closed" parabolic super- and subjets as 
denned in Section 8]. From the Crandall-Ishii lemma [5j Theorem 8.3] we 
obtain, for each r\ > 0, elements 

(a n ,p n ,X;)eV 2 S + Wl (t n ,x n ) and (b n , <f, Yf) G vi~w 2 (s n , y n ) 

such that 

a n = d t Q n (t n , x n , s n , y n ), b n = -d s Q n (t n , x n , s n , y n ), 

p n = D x Q n (t n ,x n ,s n ,y n ), q n = -D y Q n (t n , x n , s n ,y n ), 

X™ \ . -2 
J — n ^ ni 

where A n = D 2 O n (t n ,x n ,s n ,y n ); i.e., 

2 / I d / Z) 2 <^,:c™) + O(|x"-Xo| 2 ) 

In view of the sub- and supersolution properties of w\ and w 2 , the fact that 
(s n ,y n ) € [0, T) x for n large, and Assumption [Ej we deduce that 

A„ :=p(w 1 (t n ,x n )-w 2 (s n ,y n )) 

< 2( t » - t ) + W\ x") + d s cf)(s n , y n ) 
+ a\x n - y n \{l + \q n \+ en 2 \x n - y n \) 
+ a((l + \x n \)\ q n -p n \ + (l + \x n \ 2 )\Q n \), 



where 



or 



D 2 <j>(t n ,x n ) + O(\x n -x \ 2 ) 

D 2 (f)(s n ,y n ) 



By the definitions of p n and q n , it follows that 

A n < 2(t n - t ) + W\ x n ) + d s <f>(s n , y n ) 

+ a\x n - y n \(l + \D<f>{s n , y n )\ + en 2 \x n + ^n" 1 ) - y n \ + en 2 \x n - y n \) 

+ a(l + K|)(4|^ - x | 3 + |2ty(t B ,s B )| + \Dcf>(s n , y n )\) 

+ a(l + |s»f )(|Z> 2 0(tV)| + |£ 2 <K* n ,y n )l + 0(K - x | 2 )). 

Recalling ([A~7F ]) -(|A~lj] ) , letting n — > 00 leads to 

2 



jofiui — W2){to,xo) < 2dt<f)(to,xo) + ae( liminf 1 

+ 2a((l + |x |)|^(to,^o)| + (1 + \xo\ 2 )\D 2 0(t o ,x o )\), 
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which, by (|4.19|) and the definition of m in ()A.4J) . implies 

p(w 1 -2(f>- w 2 )(to,x ) < 2m (t , x ) 

after letting e — > 0. Since k > has been chosen so that m < on [0, T] x R d , 
this contradicts (|A.3p . 

Case 2: xq E C This case is handled similarly by using 

Q n (t,x,s,y) := ^n 2 (\t-s\ 2 + \x-y\ 2 ) + \t-t \ 2 + \x-x \ 4 + (l)(t,x) + (f>(s,y). 

After taking a subsequence, the corresponding sequence of maximum points 
(t n ,x n ,s n ,y n ) n >i again converges to (to, xq, to, xq), so that x n ,y n G O for 
n large enough. The rest of the proof follows the same arguments as in 
Case 1. □ 

Lemma A. 3. Under Assumption \D\ the operator H defined in (|4.17|) sat- 
isfies Assumption^ 

Proof. The argument is standard. We first observe that 

-L u (y, q, Y v ) + L u (x,p, X r >) = (p(x, u) - fi(y, u)) 1 q + fi(x, u) T (p - q) 

+ \Y. ^(x,y^) T E^\x,y,u), 

l<i<d 

where 

E(x ' y ' tt):= (aS,«)) ^ EV:= { X -F") 

and denotes the ith column of E. Since fj, is Lipschitz continuous and has 
uniformly linear growth by Assumption ID '| we have 

(fi{x,u) - [i(y,u)) T q + p(x,u) T {p - q) < a(\x - y\\q\ + (1 + \x\)\p - q\) 

for some constant a > 0. Recall from the statement of Assumption [E] the 
condition that 

S" < A n + nA 2 n , where A n := n 2 ( h ^ ^j+Q. 

Since a is Lipschitz continuous and satisfies Assumption ID'i it follows that 

S 4 (x, y, u) T E v E*(x, y, u) < n 2 \a l (x, u) - a% u)\ 2 

+ a(l + \x\ 2 + \y\ 2 )\Q\ + 0(r ] ), 

possibly after enlarging a, where a 1 denotes the ith. column of a. We conclude 
by using the Lipschitz continuity of a and the condition that \x — y\ < 1. □ 
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Remark A. 4. We conclude with some comments on Theorem IA.1I and 
Proposition I4.12[ and related results in the literature. 

The main issue in proving comparison with state constraints is to avoid 
boundary points; i.e., that y n (see the proof of Theorem IA. 1|) ends up on 
the boundary. One classical way to ensure this, is to use a perturbation of 
\x — y\ 2 in a suitable inward direction, like the function i above. Moreover, 
this requires the supersolution to be continuous at the boundary points, 
along the direction of perturbation; cf. (|4.22|) . 

In the inner normal n(x) at the boundary point x G dO is used as 
an inward direction. In the proof, one is only close to x (cf. ()A.9p ): therefore, 
the comparison result Theorem 2.2] requires the existence of a truncated 
cone around n(x) which stays inside the domain, and the continuity of the 
subsolution along the directions that it generates. Our condition (|4.20p is 
less restrictive than the corresponding requirement in we only need 

the continuity along the curve e h4 i(s), cf. (j4.22p . rather than all lines 
in a neighborhood. The function A appears because we consider parabolic 
equations, whereas focuses on the elliptic case. 

In Proposition 14.121 we give conditions (certainly not the most general 
possible) ensuring that the value function is of class 1Z{0). They should 
be compared to |11[ Condition (A3)], which is used to verify the continuity 
assumption of Theorem 2.2]. Our conditions are stronger in the sense 
that they are imposed around the boundary and not only at the boundary; 
on the other hand, we require C 1 -regularity of the boundary whereas 
requires C 3 . 

In |10] . a slightly different technique is used, based on ideas from [9|. 
First, it is assumed that at each boundary point x, there exists a fixed control 
which kills the volatility at the neighboring boundary points and keeps a 
truncated cone around the drift in the domain. This is similar to our ()4.23|) . 
except that our control is not fixed; on the other hand, we assume that it 
kills the volatility in a neighborhood of x. Thus, the conditions in |10] are 
not directly comparable to ours; e.g., if O is the unit disk in M 2 , U = [-1,1], 
fi(x,u) = —x and a(x,u) = \x 2 — u\l2, then ()4.23p is satisfied (with 5 given 
by the Euclidean distance near the boundary and u(x) = x 2 ), while [101 
Condition (2.1)] is not. Second, in |10] . the state constraint problem is 
transformed so as to introduce a Neumann-type boundary condition and 
construct a suitable test function which, as a function of its first component, 
turns out to be a uniformly strict supersolution of the Neumann boundary 
condition. The construction in |10j heavily relies on the assumption that the 
coefficients are bounded; cf. the beginning of |1Q[ Section 3] and the proof 
of [HH Theorem 3.1]. 
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